Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

transform: luaxform transform script #12211

Closed
wants to merge 7 commits into from
Closed

transform: luaxform transform script #12211

wants to merge 7 commits into from

Conversation

jlucovsky
Copy link
Contributor

Continuation of #12187

Add the luaxform transform that uses a Lua script to receive and transform a sticky buffer.

Lua transforms receive arguments

  • (always): length of input buffer and input buffer
  • (always): number of additional arguments (may be 0)
  • (when present): argument array (bounded by argument count)

Arguments are passed to the Lua script as they appear in the rule:

 luaxform:./lua/lua-transform.lua, bytes 4, offset 13, hash sha256;

When the transform is invoked, 3 additional arguments are passed (arg count will be 3)

  • bytes 4
  • offset 13
  • hash sha256

The Lua script is responsible for parsing and using the arguments and returning a tuple: output-buffer, output-buffer-byte-count

I used this Lua script:

function init (args)
  -- Optional, no return value
end

function transform(input_len, input, argc, args)
    SCLogNotice("Input length: " .. input_len .. " arg-count: " .. argc)
    for i = 1, argc do
        print("Argument " .. i .. ":", args[i])
    end
    return string.upper(input), input_len
end

return 0

Example rule using the Lua script:

alert http any any -> any any (msg:"Lua Xform example"; flow:established;  file.data; luaxform:./lua/lua-transform.lua; content: "abc"; sid: 1;)

Link to ticket: https://redmine.openinfosecfoundation.org/issues/2290

Describe changes:

  • Luaxform transform changes

Updates

  • Rebase

Provide values to any of the below to override the defaults.

  • To use an LibHTP, Suricata-Verify or Suricata-Update pull request,
    link to the pull request in the respective _BRANCH variable.
  • Leave unused overrides blank or remove.

SV_REPO=
SV_BRANCH=OISF/suricata-verify#2090
SU_REPO=
SU_BRANCH=
LIBHTP_REPO=
LIBHTP_BRANCH=

@jlucovsky
transforms: Make ctx available
92e5264 
This commit makes the detection engine thread context available for
transforms to use. The Lua transform requires this value.

Issue: 2290
@jlucovsky
hash: Provide detection engine context to free fn
d138734 
Issue: 2290

This commit extends the hash table logic with an alternate free function
that provides the detection engine context.

Users that wish to use the next functionality must use the
HashListTableInitWithCtx function when initializing the hash table.
Using this interface will result in the hash table "free with context"
function (new) being used instead.
@jlucovsky
detect/engine: Prevent double-free of keyword hash
94a1437 
Issue: 2290

Defer freeing the keyword hash table until the engine context has
been freed. This eliminates a double-free from occurring.

For the unittests ONLY, clear the keyword_hash to prevent a double
free attempt.
@jlucovsky
transform: Add luaxform files
e2255be 
Issue: 2290

This commit adds the source files for the new transform -- luaxform.
@jlucovsky
doc: Document luaxform transform
d3197ee 
Issue: 2290
@jlucovsky
detect/transform: Refactor setup/apply pattern
e73a2ea 
git grep -A 1 -w InspectionBufferSetup shows numbers cases of the pattern:
    - InspectionBufferSetup
    - InspectionBufferApplyTransforms

Refactor the implementations of those functions into
InspectionBufferSetupAndApplyTransforms to reduce function call count.

Issuer: 2290
@jlucovsky
clang/format: Formatting fixups
7001a86
@jlucovsky jlucovsky mentioned this pull request Dec 3, 2024
Closed
@codecov Codecov
Copy link

codecov bot commented Dec 3, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 86.93694% with 58 lines in your changes missing coverage. Please review.

Project coverage is 83.17%. Comparing base (e9173f3) to head (7001a86).
Report is 7 commits behind head on master.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master   #12211      +/-   ##
==========================================
- Coverage   83.17%   83.17%   -0.01%     
==========================================
  Files         912      914       +2     
  Lines      257111   257325     +214     
==========================================
+ Hits       213856   214025     +169     
- Misses      43255    43300      +45
Flag Coverage Δ
fuzzcorpus 60.94% <47.22%> (-0.07%) ⬇️
livemode 19.41% <13.19%> (+<0.01%) ⬆️
pcap 44.29% <30.32%> (-0.10%) ⬇️
suricata-verify 62.79% <84.49%> (+0.02%) ⬆️
unittests 59.14% <26.57%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@suricata-qa
Copy link

Information: QA ran without warnings.

Pipeline 23651

@jufajardini jufajardini added the needs rebase Needs rebase to master label Dec 5, 2024
jufajardini
jufajardini requested changes Dec 5, 2024
View reviewed changes
Copy link
Contributor

@jufajardini jufajardini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for taking so long to get back to reviewing this... >_<

doc/userguide/lua/lua-usage.rst Show resolved Hide resolved
doc/userguide/rules/lua-detection.rst Show resolved Hide resolved
doc/userguide/rules/transforms.rst Show resolved Hide resolved
doc/userguide/rules/transforms.rst Show resolved Hide resolved
doc/userguide/rules/transforms.rst Show resolved Hide resolved
doc/userguide/rules/transforms.rst Show resolved Hide resolved
@jlucovsky jlucovsky mentioned this pull request Dec 8, 2024
Closed
@jlucovsky
Copy link
Contributor Author

Continued in #12246

@jlucovsky jlucovsky closed this Dec 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jufajardini jufajardini jufajardini requested changes

@jasonish jasonish Awaiting requested review from jasonish jasonish is a code owner

@victorjulien victorjulien Awaiting requested review from victorjulien victorjulien is a code owner

Assignees
No one assigned
Labels
needs rebase Needs rebase to master
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jlucovsky @suricata-qa @jufajardini